1. Field of the Invention
The present invention relates to a color filter array, an imaging device, and an image processing unit used for an image capturing apparatus, such as a digital camera using a solid-state imaging device.
2. Description of the Related Art
Single-plate color image capturing apparatuses that include an image processing unit are known. In the image capturing apparatuses, a color filter is bonded to each of a plurality of pixels of a single-plate solid-state imaging device and the image processing unit allocates all colors to the position of each pixel using a mosaic image of colors captured by the imaging device.
In such image capturing apparatuses using a single-plate solid-state imaging device, only a single spectral sensitivity is obtained. Accordingly, in general, to obtain a color image, different color filters are bonded to a plurality of pixels so as to be arranged in a specific pattern. In the captured image, each pixel provides only one color. Therefore, in terms of colors, a mosaic image is generated. However, by interpolating the color of a pixel using color information that can be obtained from the adjacent pixels, an image in which each pixel has a full color can be generated. Such an interpolating process is referred to as a “color separation process” or a “demosaic process”.
For most of the single-plate color image capturing apparatuses, as a color filter arrangement, a Bayer array format described in Bryce E. Bayer, U.S. Pat. No. 3,971,065 entitled “COLOR IMAGING ARRAY” is used. As shown in FIG. 1, in this arrangement, G color filters are arranged in a checkered pattern so that the density of the G color filters is twice that of R color filters or B color filters. Accordingly, in most methods of interpolating a color of a pixel, a G color having a large amount of information and a strong relation to the brightness of light is interpolated to all the pixels first, and subsequently, R and B colors are interpolated using the G color as a reference.
For example, in the method described in Ozawa, Akiyama, Satoh, Nagahara and Miura, U.S. Pat. No. 4,716,455 entitled “Chrominance Signal Interpolation Device for a Color Camera,” on ground that a ratio of a low-frequency component of each color in a local region is substantially constant, G color is allocated to all of the pixels. Thereafter, an average of a ratio of R color to G color and an average of a ratio of B color to G color of the adjacent pixels are multiplied by the G color of a pixel of interest. Thus, unknown color components are estimated.
In addition, in this interpolation method, on ground that the G color filters are arranged in a checkered pattern, the resolutions in a horizontal direction and a vertical direction can be increased.
For example, Japanese Patent No. 2931520 entitled “Color Separation Circuit of Single-plate Color Video Camera” describes a technique in which a correlation value at the position of an interpolated pixel in a horizontal direction or in a vertical direction is computed. Two interpolation values obtained by processing means appropriate when the correlation in the horizontal direction is strong and processing means appropriate when the correlation in the vertical direction is strong are mixed using the correlation value.
According to the above-described techniques, in a single-plate color image capturing apparatus, the R, G, and B colors can be allocated to positions of all the pixels for high-resolution display.
However, since each color is discretely sampled, aliasing (overlap of a high-frequency component with lower frequency components) occurs when a captured image contains a high-frequency component having a frequency higher than the Nyquist frequency, and therefore, a color different from an original color is estimated.
This color is referred to as a “false color”. The false color is noticeable when a color filter arrangement in which color filters are regularly arranged is used, since a significant overlap occurs in a specific spatial frequency range. Once a false color is generated, it cannot be determined whether a color is an original color having an original low frequency or the false color caused by the overlap of a high-frequency component. Therefore, the false color cannot be removed by using a frequency filter.
Accordingly, to reduce the occurrence of a false color, known single-plate color image capturing apparatuses need to include an optical low-pass filter disposed in front of the imaging device so as to remove a high-frequency component in advance. However, the optical low-pass filter does not have a sharp cut-off capability for the Nyquist frequency. Therefore, if the single-plate color image capturing apparatuses attempt to completely prevent the occurrence of the false color, low-frequency components having a frequency lower than the Nyquist frequency could also be cut off.
In addition, in the Bayer arrangement, the Nyquist frequency of the R signal or the B signal is lower than that of the G signal. Accordingly, an optical low-pass filter suitable for the Nyquist frequency of the R channel or the B channel decreases the resolution of the G channel.
In practical applications, since a decrease in resolution is not allowed, complete removal of the false color is difficult. Furthermore, the installation of an optical low-pass filter prevents miniaturization and cost reduction of the image capturing apparatus.
Additionally, in contrast to the Bayer arrangement of three RGB color filters, the fidelity and the dynamic range of colors can be increased by using a filter arrangement of four colors or more.
For accurate color reproduction, a method using a large number of filters each transmitting light only in a narrow wavelength range, as shown in FIG. 2A, is more suitable than a method using a small number of filters each transmitting light in a wide wavelength range, as shown in FIG. 2C.
For an increase in the dynamic range, a method using a plurality of filters that have different transmittances but transmit light in the same wavelength range, as shown in FIG. 2B, is more suitable than a method using the filters shown in FIG. 2C.
However, the Bayer arrangement is still widely used. This is because as the number of pixels for one color is decreased, the resolution of that color channel deteriorates, and therefore, a false color easily occurs.
To solve this problem, technology has been invented in which the regularity of the color filter arrangement is reduced in order to reduce the occurrence of a false color.
More precisely, this technology solves the following problem. That is, a false color is visually noticeable and removal of the false color is difficult if most of the false colors occur in a specific spatial frequency range.
Similarly, as used herein, the reduction in the occurrence of a false color refers to the reduction in the occurrence of a false color concentrated in a specific spatial frequency range.
In a pseudo-random Bayer arrangement introduced by FillFactory, Belgium, (this document is available at http://www.fillfactory.com/htm/technology/htm/rgbfaq.htm), G color filters are arranged in a checkered pattern. In addition, at positions other than those of the G color filters, R and B color filters are pseudo-randomly arranged. This arrangement is referred to as a “three-color G-checkered pseudo-random arrangement”.
Additionally, Japanese Unexamined Patent Application Publication No. 2000-316169 describes a six-color random arrangement in which four sides or four corners of a pixel of interest are adjacent to filters of six colors.
Furthermore, Mutze, Ulrich, Dr., EP Patent Publication No. 0,804,037 entitled “Process and system for generating a full color image or multispectral image from the image data of a CCD image sensor with a mosaic color filter” describes an arrangement including a five-color 3-by-3 repetition pattern and a pseudo-random pattern.
All of the above-described arrangements include a random pattern. In addition, the two arrangements described in Japanese Unexamined Patent Application Publication No. 2000-316169 and EP Patent Publication No. 0,804,037, (A2) employ filters of more than three colors.
Because of the random pattern in the arrangements, the false color is dispersed in a variety of spatial frequency ranges, and therefore, the false color is not noticeable. In addition, the increase in the number of filters improves the dynamic range and the performance of the color reproduction.
However, although the pseudo-random Bayer arrangement introduced by FillFactory has a pseudo-random pattern, only the positions at which a false color occurs in the spatial frequency range are slightly different from those in the Bayer arrangement. This is because the frequency of the repetition is low. Therefore, in practice, the pseudo-random Bayer arrangement reduces the occurrence of a false color little. In addition, since the pseudo-random Bayer arrangement is a three-color filter arrangement, the performance of color reproduction and the dynamic range are substantially the same as those of the Bayer arrangement.
Since the two arrangements described in Japanese Unexamined Patent Application. Publication No. 2000-316169 and EP Patent Publication No. 0,804,037, (A2) employ a stronger random pattern than the Bayer arrangement or the pseudo-random Bayer arrangement, the occurrence of a false color is reduced compared with the pseudo-random Bayer arrangement or the pseudo-random Bayer arrangement. However, since, in the two arrangements, all the color filters are randomly arranged, the resolution is decreased compared with that of the Bayer arrangement or the pseudo-random Bayer arrangement.
In general, in color separation processes, one color signal is interpolated with a high resolution first. Thereafter, the other color signals are interpolated using that color signal as a reference. Accordingly, compared with the Bayer arrangement or the pseudo-random Bayer arrangement in which G color filters are arranged in a checkered pattern and a correlation process described in Japanese Patent No. 2931520 is used, it is very difficult for the arrangements described in Japanese Unexamined Patent Application Publication No. 2000-316169 and EP Patent Publication No. 0,804,037, (A2) to generate a reference color. Consequently, the reproducible frequency range is significantly different for the position of each pixel.
Furthermore, since the arrangements described in Japanese Unexamined Patent Application Publication No. 2000-316169 and EP Patent Publication No. 0,804,037, (A2) increase the number of colors compared with the Bayer arrangement, the number of pixels for one color is reduced. This results in a further decrease in resolution.
Still furthermore, in general, to read a signal out of a solid-state imaging device at high speed, the signals from the pixels are thinned out (dumped) or summed. The dumping and summing processes are cyclically executed. Accordingly, if a random filter arrangement is used, a filter pattern after dumping may be changed from the original pattern or signals from different pixels may be summed.
As noted above, while a random pattern in the filter arrangement and the increase in the number of colors reduce the occurrence of a false color and increase the dynamic range and the performance of color reproduction, the random pattern and the increase in the number of colors decrease the resolution and cause an unsuccessful operation of dumping and summing the signals from the pixels.